{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistics回归-Pima Indians Diabetes Data Set（皮马印第安人糖尿病数据集） "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "一、\t数据说明："
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "根据现有的医疗信息预测5年内皮马印第安人糖尿病发作的概率。\n",
    "数据链接：https://archive.ics.uci.edu/ml/datasets/Pima+Indians+Diabetes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "字段说明：\n",
    "\n",
    "数据集共9个字段: \n",
    "    \n",
    "pregnants：怀孕次数\n",
    "\n",
    "Plasma_glucose_concentration：口服葡萄糖耐量试验中2小时后的血浆葡萄糖浓度\n",
    "\n",
    "blood_pressure：舒张压，单位:mm Hg\n",
    "    \n",
    "Triceps_skin_fold_thickness：三头肌皮褶厚度，单位：mm\n",
    "\n",
    "serum_insulin：餐后血清胰岛素，单位:mm\n",
    "    \n",
    "BMI：体重指数（体重（公斤）/ 身高（米）^2）\n",
    "\n",
    "Diabetes_pedigree_function：糖尿病家系作用\n",
    "\n",
    "Age：年龄\n",
    "\n",
    "Target：标签， 0表示不发病，1表示发病\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "#import需要的模块\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.model_selection import GridSearchCV"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 评价指标一：logloss"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import log_loss\n",
    "from matplotlib import pyplot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 读取数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pregnants</th>\n",
       "      <th>Plasma_glucose_concentration</th>\n",
       "      <th>blood_pressure</th>\n",
       "      <th>Triceps_skin_fold_thickness</th>\n",
       "      <th>serum_insulin</th>\n",
       "      <th>BMI</th>\n",
       "      <th>Diabetes_pedigree_function</th>\n",
       "      <th>Age</th>\n",
       "      <th>Target</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.639947</td>\n",
       "      <td>0.866045</td>\n",
       "      <td>-0.031990</td>\n",
       "      <td>0.670643</td>\n",
       "      <td>-0.181541</td>\n",
       "      <td>0.166619</td>\n",
       "      <td>0.468492</td>\n",
       "      <td>1.425995</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.844885</td>\n",
       "      <td>-1.205066</td>\n",
       "      <td>-0.528319</td>\n",
       "      <td>-0.012301</td>\n",
       "      <td>-0.181541</td>\n",
       "      <td>-0.852200</td>\n",
       "      <td>-0.365061</td>\n",
       "      <td>-0.190672</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1.233880</td>\n",
       "      <td>2.016662</td>\n",
       "      <td>-0.693761</td>\n",
       "      <td>-0.012301</td>\n",
       "      <td>-0.181541</td>\n",
       "      <td>-1.332500</td>\n",
       "      <td>0.604397</td>\n",
       "      <td>-0.105584</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.844885</td>\n",
       "      <td>-1.073567</td>\n",
       "      <td>-0.528319</td>\n",
       "      <td>-0.695245</td>\n",
       "      <td>-0.540642</td>\n",
       "      <td>-0.633881</td>\n",
       "      <td>-0.920763</td>\n",
       "      <td>-1.041549</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-1.141852</td>\n",
       "      <td>0.504422</td>\n",
       "      <td>-2.679076</td>\n",
       "      <td>0.670643</td>\n",
       "      <td>0.316566</td>\n",
       "      <td>1.549303</td>\n",
       "      <td>5.484909</td>\n",
       "      <td>-0.020496</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   pregnants  Plasma_glucose_concentration  blood_pressure  \\\n",
       "0   0.639947                      0.866045       -0.031990   \n",
       "1  -0.844885                     -1.205066       -0.528319   \n",
       "2   1.233880                      2.016662       -0.693761   \n",
       "3  -0.844885                     -1.073567       -0.528319   \n",
       "4  -1.141852                      0.504422       -2.679076   \n",
       "\n",
       "   Triceps_skin_fold_thickness  serum_insulin       BMI  \\\n",
       "0                     0.670643      -0.181541  0.166619   \n",
       "1                    -0.012301      -0.181541 -0.852200   \n",
       "2                    -0.012301      -0.181541 -1.332500   \n",
       "3                    -0.695245      -0.540642 -0.633881   \n",
       "4                     0.670643       0.316566  1.549303   \n",
       "\n",
       "   Diabetes_pedigree_function       Age  Target  \n",
       "0                    0.468492  1.425995       1  \n",
       "1                   -0.365061 -0.190672       0  \n",
       "2                    0.604397 -0.105584       1  \n",
       "3                   -0.920763 -1.041549       0  \n",
       "4                    5.484909 -0.020496       1  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train = pd.read_csv(\"FE_pima-indians-diabetes.csv\")\n",
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 768 entries, 0 to 767\n",
      "Data columns (total 9 columns):\n",
      "pregnants                       768 non-null float64\n",
      "Plasma_glucose_concentration    768 non-null float64\n",
      "blood_pressure                  768 non-null float64\n",
      "Triceps_skin_fold_thickness     768 non-null float64\n",
      "serum_insulin                   768 non-null float64\n",
      "BMI                             768 non-null float64\n",
      "Diabetes_pedigree_function      768 non-null float64\n",
      "Age                             768 non-null float64\n",
      "Target                          768 non-null int64\n",
      "dtypes: float64(8), int64(1)\n",
      "memory usage: 54.1 KB\n"
     ]
    }
   ],
   "source": [
    "train.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train = train['Target']\n",
    "X_train = train.drop([\"Target\"],axis=1)\n",
    "#记录特征的名字，用于可视化\n",
    "feat_names = X_train.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 模型训练"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 缺省参数的logistics regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression\n",
    "lr = LogisticRegression()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "logloss of each fold is: [0.48797856 0.53011593 0.4562292  0.422546   0.48392885]\n",
      "cv logloss is : 0.47615970944434044\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score\n",
    "loss = cross_val_score(lr,X_train,y_train,cv=5,scoring='neg_log_loss')\n",
    "print(\"logloss of each fold is:\", -loss)\n",
    "print(\"cv logloss is :\", -loss.mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 正则化的Logistics Regression及参数调优"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 需要调休的超参数有C和正则项"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 5 folds for each of 14 candidates, totalling 70 fits\n",
      "[CV] C=0.001, penalty=l1 .............................................\n",
      "[CV] ... C=0.001, penalty=l1, score=-0.6931471805599453, total=   0.0s\n",
      "[CV] C=0.001, penalty=l1 .............................................\n",
      "[CV] ... C=0.001, penalty=l1, score=-0.6931471805599453, total=   0.0s\n",
      "[CV] C=0.001, penalty=l1 .............................................\n",
      "[CV] ... C=0.001, penalty=l1, score=-0.6931471805599453, total=   0.0s\n",
      "[CV] C=0.001, penalty=l1 .............................................\n",
      "[CV] ... C=0.001, penalty=l1, score=-0.6931471805599453, total=   0.0s\n",
      "[CV] C=0.001, penalty=l1 .............................................\n",
      "[CV] ... C=0.001, penalty=l1, score=-0.6931471805599453, total=   0.0s\n",
      "[CV] C=0.001, penalty=l2 .............................................\n",
      "[CV] ... C=0.001, penalty=l2, score=-0.6252332751246228, total=   0.0s\n",
      "[CV] C=0.001, penalty=l2 .............................................\n",
      "[CV] ... C=0.001, penalty=l2, score=-0.6338686001564596, total=   0.0s\n",
      "[CV] C=0.001, penalty=l2 .............................................\n",
      "[CV] .... C=0.001, penalty=l2, score=-0.625661213962723, total=   0.0s\n",
      "[CV] C=0.001, penalty=l2 .............................................\n",
      "[CV] ... C=0.001, penalty=l2, score=-0.6208734221572547, total=   0.0s\n",
      "[CV] C=0.001, penalty=l2 .............................................\n",
      "[CV] ... C=0.001, penalty=l2, score=-0.6326871610823708, total=   0.0s\n",
      "[CV] C=0.01, penalty=l1 ..............................................\n",
      "[CV] .... C=0.01, penalty=l1, score=-0.6359066018003606, total=   0.0s\n",
      "[CV] C=0.01, penalty=l1 ..............................................\n",
      "[CV] .... C=0.01, penalty=l1, score=-0.6446073600721886, total=   0.0s\n",
      "[CV] C=0.01, penalty=l1 ..............................................\n",
      "[CV] .... C=0.01, penalty=l1, score=-0.6332810905191685, total=   0.0s\n",
      "[CV] C=0.01, penalty=l1 ..............................................\n",
      "[CV] .... C=0.01, penalty=l1, score=-0.6302418522346175, total=   0.0s\n",
      "[CV] C=0.01, penalty=l1 ..............................................\n",
      "[CV] .... C=0.01, penalty=l1, score=-0.6358010482921886, total=   0.0s\n",
      "[CV] C=0.01, penalty=l2 ..............................................\n",
      "[CV] .... C=0.01, penalty=l2, score=-0.5120767708654068, total=   0.0s\n",
      "[CV] C=0.01, penalty=l2 ..............................................\n",
      "[CV] ..... C=0.01, penalty=l2, score=-0.543227572433324, total=   0.0s\n",
      "[CV] C=0.01, penalty=l2 ..............................................\n",
      "[CV] .... C=0.01, penalty=l2, score=-0.5067380443593053, total=   0.0s\n",
      "[CV] C=0.01, penalty=l2 ..............................................\n",
      "[CV] .... C=0.01, penalty=l2, score=-0.4873125612462518, total=   0.0s\n",
      "[CV] C=0.01, penalty=l2 ..............................................\n",
      "[CV] .... C=0.01, penalty=l2, score=-0.5253888382730234, total=   0.0s\n",
      "[CV] C=0.1, penalty=l1 ...............................................\n",
      "[CV] .... C=0.1, penalty=l1, score=-0.48922344785236926, total=   0.0s\n",
      "[CV] C=0.1, penalty=l1 ...............................................\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   3 out of   3 | elapsed:    0.0s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   4 out of   4 | elapsed:    0.0s remaining:    0.0s\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[CV] ..... C=0.1, penalty=l1, score=-0.5248827281780112, total=   0.0s\n",
      "[CV] C=0.1, penalty=l1 ...............................................\n",
      "[CV] .... C=0.1, penalty=l1, score=-0.45824944524958855, total=   0.0s\n",
      "[CV] C=0.1, penalty=l1 ...............................................\n",
      "[CV] ..... C=0.1, penalty=l1, score=-0.4333634233695568, total=   0.0s\n",
      "[CV] C=0.1, penalty=l1 ...............................................\n",
      "[CV] .... C=0.1, penalty=l1, score=-0.48686939828576076, total=   0.0s\n",
      "[CV] C=0.1, penalty=l2 ...............................................\n",
      "[CV] .... C=0.1, penalty=l2, score=-0.48350007746296986, total=   0.0s\n",
      "[CV] C=0.1, penalty=l2 ...............................................\n",
      "[CV] ...... C=0.1, penalty=l2, score=-0.525149433695313, total=   0.0s\n",
      "[CV] C=0.1, penalty=l2 ...............................................\n",
      "[CV] ..... C=0.1, penalty=l2, score=-0.4604874856554894, total=   0.0s\n",
      "[CV] C=0.1, penalty=l2 ...............................................\n",
      "[CV] .... C=0.1, penalty=l2, score=-0.42929826862168724, total=   0.0s\n",
      "[CV] C=0.1, penalty=l2 ...............................................\n",
      "[CV] ...... C=0.1, penalty=l2, score=-0.485373395496449, total=   0.0s\n",
      "[CV] C=1, penalty=l1 .................................................\n",
      "[CV] ...... C=1, penalty=l1, score=-0.48815770831960104, total=   0.0s\n",
      "[CV] C=1, penalty=l1 .................................................\n",
      "[CV] ........ C=1, penalty=l1, score=-0.529170015981784, total=   0.0s\n",
      "[CV] C=1, penalty=l1 .................................................\n",
      "[CV] ....... C=1, penalty=l1, score=-0.4556167764702757, total=   0.0s\n",
      "[CV] C=1, penalty=l1 .................................................\n",
      "[CV] ...... C=1, penalty=l1, score=-0.42237499954107427, total=   0.0s\n",
      "[CV] C=1, penalty=l1 .................................................\n",
      "[CV] ...... C=1, penalty=l1, score=-0.48452516295349723, total=   0.0s\n",
      "[CV] C=1, penalty=l2 .................................................\n",
      "[CV] ....... C=1, penalty=l2, score=-0.4879785610109463, total=   0.0s\n",
      "[CV] C=1, penalty=l2 .................................................\n",
      "[CV] ....... C=1, penalty=l2, score=-0.5301159331731353, total=   0.0s\n",
      "[CV] C=1, penalty=l2 .................................................\n",
      "[CV] ....... C=1, penalty=l2, score=-0.4562291976119336, total=   0.0s\n",
      "[CV] C=1, penalty=l2 .................................................\n",
      "[CV] ...... C=1, penalty=l2, score=-0.42254600245513574, total=   0.0s\n",
      "[CV] C=1, penalty=l2 .................................................\n",
      "[CV] ...... C=1, penalty=l2, score=-0.48392885297055127, total=   0.0s\n",
      "[CV] C=10, penalty=l1 ................................................\n",
      "[CV] ..... C=10, penalty=l1, score=-0.48903289677350886, total=   0.0s\n",
      "[CV] C=10, penalty=l1 ................................................\n",
      "[CV] ...... C=10, penalty=l1, score=-0.5310377604517517, total=   0.0s\n",
      "[CV] C=10, penalty=l1 ................................................\n",
      "[CV] ...... C=10, penalty=l1, score=-0.4558883899426952, total=   0.0s\n",
      "[CV] C=10, penalty=l1 ................................................\n",
      "[CV] ..... C=10, penalty=l1, score=-0.42195637300618066, total=   0.0s\n",
      "[CV] C=10, penalty=l1 ................................................\n",
      "[CV] ..... C=10, penalty=l1, score=-0.48410446070104923, total=   0.0s\n",
      "[CV] C=10, penalty=l2 ................................................\n",
      "[CV] ...... C=10, penalty=l2, score=-0.4890294879696647, total=   0.0s\n",
      "[CV] C=10, penalty=l2 ................................................\n",
      "[CV] ...... C=10, penalty=l2, score=-0.5311363222164094, total=   0.0s\n",
      "[CV] C=10, penalty=l2 ................................................\n",
      "[CV] ..... C=10, penalty=l2, score=-0.45595951416730635, total=   0.0s\n",
      "[CV] C=10, penalty=l2 ................................................\n",
      "[CV] ...... C=10, penalty=l2, score=-0.4219795332800078, total=   0.0s\n",
      "[CV] C=10, penalty=l2 ................................................\n",
      "[CV] ..... C=10, penalty=l2, score=-0.48408647346409145, total=   0.0s\n",
      "[CV] C=100, penalty=l1 ...............................................\n",
      "[CV] .... C=100, penalty=l1, score=-0.48913955009491467, total=   0.0s\n",
      "[CV] C=100, penalty=l1 ...............................................\n",
      "[CV] ..... C=100, penalty=l1, score=-0.5312344836336859, total=   0.0s\n",
      "[CV] C=100, penalty=l1 ...............................................\n",
      "[CV] .... C=100, penalty=l1, score=-0.45593094058578676, total=   0.0s\n",
      "[CV] C=100, penalty=l1 ...............................................\n",
      "[CV] .... C=100, penalty=l1, score=-0.42192546009108134, total=   0.0s\n",
      "[CV] C=100, penalty=l1 ...............................................\n",
      "[CV] ..... C=100, penalty=l1, score=-0.4841108328240825, total=   0.0s\n",
      "[CV] C=100, penalty=l2 ...............................................\n",
      "[CV] .... C=100, penalty=l2, score=-0.48914440233874235, total=   0.0s\n",
      "[CV] C=100, penalty=l2 ...............................................\n",
      "[CV] ...... C=100, penalty=l2, score=-0.531246919864776, total=   0.0s\n",
      "[CV] C=100, penalty=l2 ...............................................\n",
      "[CV] .... C=100, penalty=l2, score=-0.45593525922772243, total=   0.0s\n",
      "[CV] C=100, penalty=l2 ...............................................\n",
      "[CV] .... C=100, penalty=l2, score=-0.42192490874569916, total=   0.0s\n",
      "[CV] C=100, penalty=l2 ...............................................\n",
      "[CV] .... C=100, penalty=l2, score=-0.48410727360689465, total=   0.0s\n",
      "[CV] C=1000, penalty=l1 ..............................................\n",
      "[CV] ... C=1000, penalty=l1, score=-0.48914899584335686, total=   0.0s\n",
      "[CV] C=1000, penalty=l1 ..............................................\n",
      "[CV] .... C=1000, penalty=l1, score=-0.5312562811037194, total=   0.0s\n",
      "[CV] C=1000, penalty=l1 ..............................................\n",
      "[CV] .... C=1000, penalty=l1, score=-0.4559285957654249, total=   0.0s\n",
      "[CV] C=1000, penalty=l1 ..............................................\n",
      "[CV] ... C=1000, penalty=l1, score=-0.42192045716900545, total=   0.0s\n",
      "[CV] C=1000, penalty=l1 ..............................................\n",
      "[CV] .... C=1000, penalty=l1, score=-0.4841108115749953, total=   0.0s\n",
      "[CV] C=1000, penalty=l2 ..............................................\n",
      "[CV] .... C=1000, penalty=l2, score=-0.4891559997762149, total=   0.0s\n",
      "[CV] C=1000, penalty=l2 ..............................................\n",
      "[CV] .... C=1000, penalty=l2, score=-0.5312580709718574, total=   0.0s\n",
      "[CV] C=1000, penalty=l2 ..............................................\n",
      "[CV] .... C=1000, penalty=l2, score=-0.4559328627319946, total=   0.0s\n",
      "[CV] C=1000, penalty=l2 ..............................................\n",
      "[CV] ... C=1000, penalty=l2, score=-0.42191946802406416, total=   0.0s\n",
      "[CV] C=1000, penalty=l2 ..............................................\n",
      "[CV] ..... C=1000, penalty=l2, score=-0.484109407281101, total=   0.0s\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Done  70 out of  70 | elapsed:    0.4s finished\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=5, error_score='raise',\n",
       "       estimator=LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n",
       "          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n",
       "          verbose=0, warm_start=False),\n",
       "       fit_params=None, iid=True, n_jobs=1,\n",
       "       param_grid={'penalty': ['l1', 'l2'], 'C': [0.001, 0.01, 0.1, 1, 10, 100, 1000]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, return_train_score='warn',\n",
       "       scoring='neg_log_loss', verbose=5)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "penaltys = ['l1','l2']\n",
    "Cs = [0.001,0.01,0.1,1,10,100,1000]\n",
    "tuned_parameters = dict(penalty = penaltys,C = Cs)\n",
    "\n",
    "lr_penalty = LogisticRegression()\n",
    "grid = GridSearchCV(lr_penalty,tuned_parameters,cv=5,scoring='neg_log_loss',n_jobs=1,verbose=5)\n",
    "grid.fit(X_train,y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 打印超参数值"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.4760275754954495\n",
      "{'C': 1, 'penalty': 'l1'}\n"
     ]
    }
   ],
   "source": [
    "print(-grid.best_score_)\n",
    "print(grid.best_params_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 评价指标二：正确率"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7747395833333334\n",
      "{'C': 0.1, 'penalty': 'l2'}\n"
     ]
    }
   ],
   "source": [
    "penaltys = ['l1','l2']\n",
    "Cs = [0.001,0.01,0.1,1,10,100,1000]\n",
    "tuned_parameters = dict(penalty = penaltys,C = Cs)\n",
    "\n",
    "lr_penalty = LogisticRegression()\n",
    "\n",
    "grid = GridSearchCV(lr_penalty,tuned_parameters,cv=5)\n",
    "grid.fit(X_train,y_train)\n",
    "\n",
    "print(grid.best_score_)\n",
    "print(grid.best_params_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "l2\n",
      "0.1\n",
      "[[ 0.36167135  0.99773092 -0.0648661   0.05966521 -0.04513553  0.54600449\n",
      "   0.25990383  0.15544357]]\n"
     ]
    }
   ],
   "source": [
    "print(grid.best_params_['penalty'])\n",
    "print(grid.best_params_['C'])\n",
    "print(grid.best_estimator_.coef_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
